AWESOME: an auction and witness enhanced SLA model for decentralized cloud marketplaces

In this section, we first perform the requirement analysis using two industrial use cases and show the design objectives of the system. Then, we describe our AWESOME model in detail, including actor identification, on-chain vs. off-chain interactions, and system components & workflows.

In industrial innovations (e.g., crowd journalism and disaster early warning) and scientific applications (e.g., research data management), cloud services are playing an increasingly important role in real-time data processing (e.g., multimedia acquired by mobile devices), running simulations (e.g., for predicting possible disasters), and for enabling extensive scale collaborations (e.g., for running distributed scientific workflows). Therefore, it is necessary to employ multiple data centers or providers to handle decentralized collaboration between resource providers and customers in several industrial use cases.

Use case 1. Decentralized cloud marketplace for social media (taken from EU ARTICONF projectFootnote 4): crowd journalism for real-time news reporting during live sports, music events, or natural disasters. Individual citizen journalists make photos or videos of the news and trade them via the news platform. The system has to detect fake news from those crowdsourced contents by running real-time processing from multiple cloud providers or engaging human experts to review them.

Use case 2. Decentralized service marketplace for medical data management (taken from EU CLARIFY projectFootnote 5): sharing and utilizing pathology data provided by hospitals or individuals from different countries, where various medical data access constraints are often applied. When a machine learning application for studying breast cancer must use data from multiple hospitals, the application developer has to select cloud providers from a decentralized marketplace that meet the application needs (e.g., geolocation, capacity, and price).

We can therefore highlight the following requirements and challenges from those use cases:

Provider selection, service customization, and capacity planning challenges. The developer has to select cloud services from different providers (very often multiple ones) due to distributed data locations (e.g., sensors or repositories), various data access constraints (e.g., for medical data), and performance constraints (e.g., for real-time decisions in early warning). The various price and reputation models make the selection time-consuming and challenging to be optimal.

SLA interoperability and guarantee challenge. The time-critical application constraints, e.g., processing media contents during crowd news reporting and real-time decision-making, require the profound optimization of the application logic and the quality guarantee of the cloud services. However, the diverse SLA terms among providers and the uncertainties in the SLA guarantee make performance optimization difficult.

Difficulties in setting up incentive models and customizing witness games in a decentralized marketplace. The business logic in a decentralized marketplace is often realized by smart contracts, which are supposed to be immutable after being deployed on blockchains. However, any careless design or mistake may cause unexpected loss.

Virtual infrastructure automation challenge. When an application involves multiple providers or data centers, the provisioning of the virtual infrastructure, deployment of the software platform and application components, monitoring, and adaptation of the application need to be ideally automated. However, the diverse Application Programming Interfaces (APIs) from different providers and the interoperability issues across those providers make automated provisioning and deployment a challenge. This leads to a high level of complexity in monitoring runtime infrastructure quality, detecting SLA violations, and adapting the infrastructure when violations happen.

To tackle these challenges in a decentralized cloud marketplace, we propose the AWESOME framework. The AWESOME software architecture consists of novel technologies in DApp DevOps, game theory, blockchain, and smart contracts.

We aim to provide guidelines, tools, and templates that can aid developers in developing a DApp for a specific business that can benefit from a decentralized architecture. The system needs to provide service customers and providers a platform that can facilitate easy SLA generation and operation, and allow decentralized witnesses to monitor such SLAs. Furthermore, the system needs to be generic and modular. DApp developers who use the system could customize the model for their own business needs and adapt it to other blockchain use cases. Specifically, the objectives of the AWESOME model can be summarized as follows:

Objective 1: Improve the customer/provider selection in a decentralized ecosystem by developing an automated service auction framework to enable dynamic business relations between a consumer(s) and providers and establish SLAs.

Objective 2: Improve the service quality and SLA’s trustworthiness between consumer(s) and providers by establishing a decentralized witness mechanism to monitor the SLA violations and automate the procedure for SLA compensation and payment.

Objective 3: Improve the model usability by developing easy-to-use customizable DApp GUIs for general cloud users to interact with different smart contracts.

Objective 4: Improve the continuous DevOps of DApps by providing an integrated contract factory to improve smart contracts’ security and efficiency.

Actors which interact with the AWESOME model are human roles, external systems, or devices that exchange information with the DApp. With this in mind, we identify the following actors:

Service Customer: Service customers use the DApp to find providers that can offer them services. They should be able to make listings on the platform and sign an SLA with a service provider. They pay for these services but can receive compensation in case of SLA violations.

Service Provider: Service providers use the DApp to list their available services on the platform for auction. They earn monetary rewards for these services but may be penalized in case of SLA violations.

Auction Witness: Witnesses can use the DApp to monitor SLAs and receive monetary rewards for their efforts. The judgment from the witness committee can ensure that auctioned cloud services are delivered as agreed in SLAs.

AWESOME Operator: An AWESOME operator could modify the developed framework for a specific business use case. They need to read the provided documentation, edit the user interfaces, blockchain APIs, and smart contract templates, and then deploy a custom decentralized service marketplaceFootnote 6.

In our model, customers are motivated to receive services at a low cost, and providers are incentivized to provide high-quality services in return for service fees. Two parties complete transactions and make profits by means of customized on-chain auctions. The payoffs of both parties are always positive; otherwise, an auction will never be settled. The incentive for witnesses, on the other hand, can include the following three parts: 1) a reward for their monitoring efforts; 2) a penalty if their reports about SLA violations fail to match the results of others (based on the majority rule); and 3) a blockchain transaction fee. We specify in our model that the monitoring reward is large enough so that the witness’s payoff is always positive. In this way, witnesses are always motivated to participate in our model and earn their rewards.

After identifying system objectives and actors, we can now design the system architecture as a whole. The ABCDE DApp development method [20] suggests first dividing the system into two subsystems. Namely, smart contracts running on top of the blockchain and external systems that interact with the blockchain. The authors also recommended formalizing what transactions and data should be placed on-chain and off-chain. In practice, the information that should be included in the blockchain is that with critical trust requirements. This is because on-chain information is immutable and enforces non-repudiation. In addition, not only “big data” is not suitable for the blockchain, but even “not-tiny” data should not be stored on the blockchain. This is mainly due to cost and scalability considerations; the computing power and data storage space available on the blockchain is limited. One of the typical solutions is to store raw data off-chain due to its size while storing small critical data and hashes on-chain. In terms of computation, blockchains are not the best for complex, intensive computations; however, they provide a benefit in their interoperability properties as many systems can access it [21]. The authors of the ABCDE development methodology suggest as a general guideline that data with transparent and immutable requirements for DApps should be managed on the blockchain system [20]. In this section, we follow this idea to demonstrate our design choices.

The data and activities the system should keep on-chain include:

Auction: To support transparent trade, the auction of service tasks should be conducted on the blockchain to maintain fairness and prevent fraud. Implementing decentralized auctions on the blockchain can also avoid the cheating auctioneers of traditional centralized auctions and save commission fees.

Witness reports: In order to avoid tampering with the SLA reports, they should be placed on-chain. While this may lead to witnesses viewing each other’s reports and reporting accordingly, the reports should be transparent. One effective way to address this issue and protect privacy is to submit the hashes of the reports in a specified time window.

SLAs: It is essential to include SLA details between the service provider, customer, and witnesses in the blockchain, as all these data have trust requirements. However, since SLAs may be larger textual files that could give a large load to the blockchain, a possible solution is to place SLA metadata that can unlock the SLA off-chain while keeping the cryptographic hash on-chain.

Payment enforcement: In case of an SLA violation, a smart contract should be used to facilitate payments to providers, customers, and witnesses automatically. Blockchain cryptocurrencies can support the secure and fair enforcement of money payments.

The data and activities the system should keep off-chain include:

User interface: Due to data loads, the way in which providers, customers, and witnesses interact with the platform should mostly be off-chain.

Cloud Services. Cloud services offered and used by providers and customers should be off-chain.

Pre-monitoring communication: The platform should facilitate communications between providers and customers before entering an SLA agreement so that they can privately agree upon service and monitoring terms.

Based on the above analysis, we designed the system architecture of AWESOME. The AWESOME framework consists of four subsystems in response to the proposed objectives, as shown in Fig. 2:

DApp Graphical User Interface (DGUI) provides a flexible and customizable DApp interactive environment for different AWESOME users to connect on-chain and off-chain activities. It is designed to provide a bridge between customers, providers, and witnesses who do not have IT development knowledge and assist them in calling function interfaces between different smart contract agents. Furthermore, the usability of the AWESOME DApp is ensured through a customizable interface design for business needs. More specifically, DGUI is designed with interfaces regarding 1) auctions by customers and providers; 2) SLA monitoring activities by witnesses; and 3) DApp management and maintenance by operators.

The overall architecture of the AWESOME framework

Auction-Based Service Selection (ABSS) provides an auction-based service customer/provider selection solution. This subsystem will first diagnose the use case requirements and help the user select the most suitable auction model and algorithm to achieve desirable results. Then, the management of the auction process and the enforcement of the service fee payment (in the form of cryptocurrency) are executed on the blockchain, ensuring that the whole auction is open and trustworthy. Finally, ABSS also audits bidder candidates to prevent malicious actors from joining the auction.

Decentralized Witness Committee Management (DWCM) provides a trustworthy incentive framework to manage decentralized auction witnesses. First of all, an appropriate number of witness candidates will be selected in an unbiased way to perform off-chain monitoring of cloud SLAs. DWCM will then invoke a game theory incentive mechanism based on customized payoff functions to enable selected witnesses to make correct judgments to win more profits. The subsystem will also audit the witnesses’ reputations to reward/restrict their participation in future monitoring activities.

Smart Contract Factory Orchestration (SCFO) provides tools and APIs for AWESOME operators to set the necessary smart contracts on the blockchain. More specifically, it is responsible for automating the process of planning, provisioning blockchain infrastructure, and deploying AWESOME business smart contracts. In addition, the SCFO subsystem also monitors and diagnoses smart contracts and the underlying blockchain infrastructure at runtime to provide effective adaptation solutions.

As shown in Fig. 3, the overall workflow of the AWESOME framework can be described as the following steps (using a reverse auction example). First of all, an AWESOME operator calls the DGUI subsystem to customize and generate customer, provider, and witness user interfaces (UIs) for the current use case. The AWESOME operator then calls the SCFO subsystem to initiate a contract factory. This contract factory automatically generates the required auction, witness, and SLA smart contracts to ensure trustworthy interaction between different participants. Meanwhile, it also invokes a runtime monitor for these contracts and returns the monitoring result to the AWESOME operator. Next, an AWESOME customer invokes the UI to transfer the specific business requirement to the ABSS subsystem. Based on this requirement, an auction model is selected and configured to wait for providers to submit their bids. The decentralized service providers then start to registerer in the ABSS subsystems through their UIs. When there are enough bidder candidates, smart contracts automatically start the auction process to find qualified providers. Then, the witnesses invoke their UIs to register in the DWCM subsystem. The DWCM subsystem also performs game design and an unbiased witness screener to choose the appropriate witness to form the witness committee. Finally, the selected providers collaborate to provide cloud services, and selected witnesses monitor the SLAs to win profits. The service fee and witness fee will be paid and enforced with cryptocurrency using smart contracts when the cloud service ends.

The process flow of the AWESOME model and the related stakeholders in the decentralized service ecosystem

In the entire AWESOME workflow, DGUI provides a customizable graphical interaction environment to support user-to-user interactions in business processes. ABSS selects candidate providers through an effective auction mechanism. DWCM ensures trustworthy SLA enforcement through truth-telling witness monitoring. Finally, SCFO provides automated smart contract support for the entire process. The four subsystems form a dynamic ecosystem that provides services to AWESOME users collaboratively.